Skip to content

Create regression test utility script to automate baseline file updates and testing results#1470

Merged
BenjaminBlake-NOAA merged 14 commits intoNOAA-EMC:developfrom
clyden-noaa:baseline_update
Mar 27, 2026
Merged

Create regression test utility script to automate baseline file updates and testing results#1470
BenjaminBlake-NOAA merged 14 commits intoNOAA-EMC:developfrom
clyden-noaa:baseline_update

Conversation

@clyden-noaa
Copy link
Copy Markdown
Collaborator

@clyden-noaa clyden-noaa commented Mar 17, 2026

The utility script:

  • scans the working directory for errors and reports findings
  • compares test suite runtimes with baseline times
  • detects updated baseline files
  • copies updated baseline files to baseline directory

Also changed:

  • Fixed rt-status.py to correctly report the test suite if it is a substring of another test suite
  • Fixed rrfs baseline runtime on Orion

@clyden-noaa clyden-noaa added Ready for Review This PR is ready for code review. No Baseline Change No baseline of the UPP regression tests are made. labels Mar 17, 2026
@WenMeng-NOAA
Copy link
Copy Markdown
Collaborator

WenMeng-NOAA commented Mar 26, 2026

@clyden-noaa Your PR is ready for the UPP RTs on RDHPCS.

@clyden-noaa
Copy link
Copy Markdown
Collaborator Author

clyden-noaa commented Mar 27, 2026

@WenMeng-NOAA @BenjaminBlake-NOAA
I have finished running RTs on all RDHPCs
All RTs ran successfully and within established timing thresholds
There were no baseline changes

BenjaminBlake-NOAA
BenjaminBlake-NOAA approved these changes Mar 27, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@BenjaminBlake-NOAA BenjaminBlake-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clyden-noaa Thanks for testing. This PR is ready for merging.

@BenjaminBlake-NOAA BenjaminBlake-NOAA merged commit 4005dbe into NOAA-EMC:develop Mar 27, 2026
4 checks passed
@BenjaminBlake-NOAA BenjaminBlake-NOAA mentioned this pull request Mar 27, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@WenMeng-NOAA WenMeng-NOAA WenMeng-NOAA approved these changes

@BenjaminBlake-NOAA BenjaminBlake-NOAA BenjaminBlake-NOAA approved these changes

Assignees

No one assigned

Labels

No Baseline Change No baseline of the UPP regression tests are made. Ready for commit queue Ready for Review This PR is ready for code review.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@clyden-noaa @WenMeng-NOAA @BenjaminBlake-NOAA